I. A 




(12) IN rFRN vTigQfe APPLIC \1 ION PUBl BUT D UNDER T|f F FVT^^^OOPE RATION TREATY (PCT) 




(19) World Intellectual Property Organization 
International Bureau 

(43) International Publication Date (10) International Publication Number 

11 October 2001 (11.10.2001) PCT WO 01/75783 A l 



(51) International Patent Classification 7 : G06K 19/06, 
G06F 3/033, G06K 11/18 

(21) International Application Number: PCT/SE0 1/00604 

(22) International Filing Date: 21 March 2001 (21.03.2001) 

(25) Filing Language: Swedish 

(26) Publication Language: English 

(30) Priority Data: 

0001235-1 5 April 2000 (05.04.2000) SE 

0004132-7 10 November 2000 (10.1 1.2000) SE 

(71) Applicant (for all designated States except US): ANOTO 
AB [SE/SE]; c/o C Technologies AB, Scheelevagen 15, 
S-223 70 Lund (SE). 

(72) Inventors; and 

(75) Inventors/Applicants (j'or US only): ERICSON, Petter 
[SE/SE]; Industrigatan 2 B, S-212 14 Malmo (SE). EDSO, 
Tomas [SE/SE]; Dag Hammarskjfilds vag 4F:210, S-224 
64 Lund (SE). 



(81) Designated States (national): AE, AG, AL, AM, AT, AT 
(utility model), AU, AZ, BA, BB, BG, BR, BY, BZ, CA, 
CH, CN, CO, CR, CU, CZ, CZ (utility model), DE, DE 
(utility model), DK, DK (utility model), DM, DZ, EE, EE 
(utility model), ES, FI, FI (utility model), GB, GD, GE, GH, 
GM, HR, HU, ID, IL, IN, IS, JP, KE, KG, KP, KR, KZ, LC, 
LK, LR, LS, LT, LU, LV, MA, MD, MG, MK, MN, MW, 
MX, MZ, NO, NZ, PL, PT, RO, RU, SD, SE, SG, SI, SK, 
SK (utility model), SL, TJ, TM, TR, TT, TZ, UA, UG, US, 
UZ, VN, YU, ZA, ZW. 

(84) Designated States (regional): ARIPO patent (GH, GM, 
KE, LS, MW, MZ, SD, SL, SZ, TZ, UG, ZW), Eurasian 
patent (AM, AZ, BY, KG, KZ, MD, RU, TJ, TM), European 
patent (AT, BE, CH, CY, DE, DK, ES, FI, FR, GB, GR, IE, 
IT, LU, MC, NL, PT, SE, TR), OAPI patent (BF, BJ, CF, 
CG, CI, CM, GA, GN, GW, ML, MR, NE, SN, TD, TG). 

Published: 

— with international search report 

For two-letter codes and other abbreviations, refer to the "Guid- 
ance Notes on Codes and Abbreviations" appearing at the begin- 
ning of each regular issue of the PCT Gazette. 



(74) Agent: AWA PATENT AB; Box 51 17, S-200 71 Malmo 
(SE). 



< 

go 



(54) Title: IDENTIFICATION OF A VIRTUAL RASTER PATTERN 

(57) Abstract: A method aims to identify a virtual raster pattern in an image of a surface that is provided with a plurality of position- 
coding marks. Each mark is associated with a respective intersection of raster lines belonging to the raster pattern. By means of 
the method, the virtual raster pattern is identified via Fourier analysis of the image. A computer program product and a device for 
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IDENTIFICATION OF A VIRTUAL RASTER PATTERN 

Technical Field 

The present invention relates in general to identi- 
fication of raster patterns, in particular in connection 
with position coding on a surface that is provided with 
a plurality of position-coding marks. In particular, the 
5 invention relates to identification of a virtual raster 
pattern in an image of this surface, on which each mark 
is associated with a respective intersection of raster 
lines belonging to the virtual raster pattern. 
Background Art 

In many connections it is desirable to be able to 
10 determine an absolute position on a surface. One example 
is the digitising of drawings. Another example is the 
production of an electronic version of handwritten infor- 
mation . 

Examples of previously known devices for position 

15 determination are found in US-A-5 , 852 , 434 , where a device 
for determining an absolute position is described. The 
device comprises a writing surface which is provided with 
a position-coding pattern by means of which x-y-coordi- 
nates can be determined, a detector that can detect the 

20 position-coding pattern and a processor that can deter- 
mine the position of the detector in relation to the 
writing surface on the basis of the detected position- 
coding pattern. The device makes it possible for a user 
to enter handwritten and hand-drawn information into a 

25 computer while the information is being written or drawn 
on the writing surface. 

Three examples of position coding are given in 
US-A-5 , 852 , 434 . The first example consists of symbols, 
each of which is constructed of three concentric circles. 

30 The outermost circle represents the x-coordinate and the 
middle the y-coordinate . Both the outer circles are fur- 
ther divided into 16 parts which, depending upon whether 
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they are filled in or not, indicate different numbers. 
This means that each pair of coordinates, x, y, is coded 
by a complex symbol with a special appearance. 

In the second example, the coordinates in each point 
5 on the writing surface are indicated by means of a bar 
code stack, a bar code for the x-coordinate being indi- 
cated above a bar code for the y-coordinate . 

A third example consists of a checked pattern that 
can be used to code the x- and y-coordinates . There is, 

10 however, no explanation as to how the checked pattern is 
constructed or how it can be converted into coordinates. 

A problem with the position-coding pattern of 
US-A 5,852,434 is that it is constructed of complex sym- 
bols and the smaller the symbols are made, the more dif- 

15 ficult it becomes to produce the patterned writing sur- 
face and the greater the danger of incorrect position 
determinations, while the larger the symbols are made, 
the poorer the position resolution becomes. 

A further problem is that the processing of the 

20 detected position-coding pattern by the processor becomes 
rather complicated, due to the fact that complex symbols 
are to be interpreted. 

Yet another problem is that the sensor must be 
designed in such a way that it can record four symbols at 

25 the same time, so that it is certain to include at least 
one symbol in its entirety, which is necessary in order 
for the position determination to be carried out. The 
ratio between the required sensor area and the area of 
the position-coding pattern that defines a position is 

30 thus large. 

In the international Patent Application 
PCT/SE00/01895 , which is assigned to the present Appli- 
cant, a position code is described which solves the 
above-mentioned problems. The position code consists of 

35 a raster and marks which are positioned at each raster 
point. The marks are preferably essentially the same 
size, round and displaced in relation to the raster 
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points in one of four orthogonal directions. The raster 
is virtual and is thus invisible both to the eye and to 
sensors . 

In order to decode the above-mentioned position 
5 code, it is necessary for the virtual raster to be iden- 
tified. The identification of the raster is the object of 
the present invention. 
Summary of the Invention 

The object of the present invention is thus to indi- 
cate a method for identifying a virtual raster pattern in 
10 an image of the type described by way of introduction. 

A special object is to make possible the identifi- 
cation of a regular virtual raster pattern in an image 
that is recorded with an unknown rotation and/or an 
unknown perspective between the imaging sensor and the 
15 imaged surface. 

These and other objects which will be apparent from 
the follOwiny desux-iplion are achieved completely or par- 
tially by a method according to claims 1 and 26, a com- 
puter program product according to claim 2 7 and a device 
20 for position determination according to claim 28. Prefer- 
red embodiments are defined in the subclaims. 

According to the invention, some form of Fourier 
analysis is used for identification of the virtual raster 
pattern. This provides a plurality of important advan- 
2 5 tages. During the Fourier analysis, the whole image or a 
subset thereof is processed as a single unit. In this way 
identification can be implemented with low sensitivity to 
local interference, for example noise in the image or 
dirt on the imaged surface. The use of Fourier analysis 
30 also makes possible effective and unambiguous compensa- 
tion for an unknown rotation and/or an unknown perspec- 
tive between the imaging sensor and the imaged surface. 

The above-mentioned Fourier analysis comprises clas- 
sical Fourier analysis, which, however, is normally very 
35 calculation-intensive and is often replaced by FFT, Fast 
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Fourier Transformation, which can also be used in the 
present invention. 

According to a preferred embodiment, before the 
Fourier analysis of the image, a conversion is carried 
5 out of the image into a set of discrete unit pulses that 
are placed at the positions of the marks in the image. 
Thus a classical Fourier analysis can be used, which is 
simplified as far as calculation is concerned. A double 
integral is thereby replaced by a sum of the discrete 

10 unit pulses, whereby the number of operations required is 
reduced. The identification via Fourier analysis can thus 
be implemented in an efficient way, as far as calculation 
is concerned, and can be implemented in a rapid program 
code which can be executed in real time by a processor 

15 that draws little current. 

Each unit pulse is preferably placed at the centre 
of gravity of the corresponding mark. This means that 
each mark cor rs spends to a single location which is unam- 
biguous and essentially independent of the shape of the 

20 mark. This minimises the effect of blur in the image 

caused by movement or the image not being in the focus 
of the optical system. 

According to a preferred embodiment, the Fourier 
analysis comprises the steps of calculating a spatial 

25 frequency spectrum in two dimensions on the basis of the 
image, of identifying at least two main vectors in the 
image, based on the frequency spectrum, and identifying 
the raster lines of the raster pattern on the basis of 
the main vectors. The main vectors resulting from the 

30 Fourier analysis represent the dominant directions in the 
image, in particular the perpendiculars to the directions 
of the raster lines, and the lengths of the main vectors 
correspond to the image's dominant spatial frequencies 
along the main vectors. In an orthogonal uniform raster 

3 5 pattern, the main vectors are thus two mutually orthogo- 
nal vectors with a length corresponding to the distances 
between the raster lines. 
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It should be noted that within the scope of the 
invention, the term "spatial frequency spectrum" also 
comprises its inverse, that is a "spatial wavelength 
spectrum" . 

5 It is preferable for the spatial frequency spectrum 

to be calculated on the basis of a central part of the 
image. The adverse effect of any perspective in the image 
is thereby minimised. If the image contains a perspec- 
tive, the spatial frequencies will vary across the image, 

10 and in particular in the peripheral parts thereof. In 

addition, the calculation time is reduced, since a subset 
of the image is evaluated. In certain cases, the choice 
of the central part also results in lower requirements 
concerning image quality and/or illumination of the sur- 

15 face that is being imaged. 

According to a preferred embodiment, the main vec- 
tors are identified by localising in the spatial fre- 
quency spectrum positions of peaks that: exceed a given 
threshold value, and selecting main vectors on the basis 

20 of these positions. 

It should also be noted that, even if it is pos- 
sible, it is not necessary to calculate a complete spa- 
tial frequency spectrum in two dimensions, that is for 
all possible directions and spatial frequencies in the 

25 image. The spatial frequency spectrum is calculated or 

"sampled", however, preferably based on a two-dimensional 
Fourier transform along at least two directions in the 
image . 

According to an additional preferred embodiment, the 
30 calculation of the spatial frequency spectrum and identi- 
fication of the main vectors therein are carried out by 
changing the direction of a direction vector in steps 
within an angle range, calculating at least one abso- 
lute value of the two-dimensional Fourier transform for 
35 the image on the basis of each such direction vector, 
and identifying the absolute values that exceed said 
threshold value. The angle range, which is typically 
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180° or less since a larger angle range results in 



redundant information, is searched typically in steps 
of approx. 2°-4°, although smaller or larger steps can 
be used in certain applications. The threshold value 
5 is typically 30-70% of the theoretical maximum absolute 
value, which is proportional to the number of marks in 
the subset of the image that is subjected to the Fourier 
analysis. The maximum absolute value can thus be calcu- 
lated for each subset. 

10 It is preferable that, while searching through 

the angle range, the length of the direction vector 
is changed within a frequency range that comprises the 
nominal spatial frequency of the raster pattern, that is 
the spatial frequency of the raster pattern on the imaged 

15 surface. The frequency range is suitably selected so that 
it contains all possible spatial frequencies that can 
arise as a result of the imaging relationship (rotation/ 
perspective) between the imaging sensor and the imaged 
surface. For an image in the form of a set of unit 

20 pulses, the calculation of the Fourier transform for each 
new length of a given direction vector involves only one 
extra addition step, which can be carried out in a way 
that is efficient as far as calculation is concerned. The 
length of the direction vector is preferably changed in 

25 steps that are inversely proportional to some suitable 

power of 2, for example eighths (2~ 3 ) , as the step length 
can then be calculated by a time-efficient bit shift. It 
is possible that the stepwise change of the length of the 
direction vector is discontinued upon the identification 

30 of one or more absolute values that exceed the threshold 
value, and that the direction of the direction vector is 
changed within the angle range for calculation of a new 
absolute value of the two-dimensional Fourier transform. 



35 ised by calculation of the centre of gravity of the abso- 
lute values that exceed the threshold value and that are 
adjacent to each other in the spatial frequency spectrum. 



The position of each of the peaks is suitably local- 
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This provides a relatively accurate position determina- 
tion for the peaks, even with coarse searching of the 
angle range. 

According to a preferred embodiment, the partial 
5 step of selecting at least two main vectors comprises 
having each peak position identify a candidate vector, 
having at least one current image transform, which pro- 
vides a given change in the relationship between two vec- 
tors, operate on the candidate vectors, and selecting as 

10 main vectors the candidate vectors that attain a required 
mutual relationship for said at least one current image 
transform. This partial step is implemented in order to 
establish which of the localised peak positions repre- 
sent main vectors. The peak positions can, in addition to 

15 main vectors, represent second order dominant patterns in 
the image, for example, diagonal patterns, or patterns 
resulting from interference/noise. 

The image transform is suitably a transform, for 
example elliptical, that changes both the length ratio 

20 and the angle ratio between the vectors. In this case, 
the required mutual relationship is a length and angle 
ratio which corresponds to the relationship between the 
raster lines in the original raster pattern, that is the 
virtual raster pattern on the imaged surface. For exam- 

2 5 pie, if the original raster pattern is an orthogonal uni- 
form raster pattern, the image transform should thus 
transform the main vectors to an orthogonal angle ratio 
with the length ratio 1:1. If one of the candidate vec- 
tors corresponds to a diagonal direction in the image, 

30 then the angle ratio should thus be 45° and the length 
ratio should be 

It is possible to have a series of different current 
image transforms operate sequentially on the candidate 
vectors, at least until a required mutual relationship 

35 is obtained between the candidate vectors. Alternatively, 
the current image transform can be selected adaptively, 
more particularly on the basis of an earlier image trans- 
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form that gave rise to the required relationship for a 
previous image . 

It is preferable that each current image transform 
corresponds to a given imaging relationship between the 
5 imaging sensor and the imaged surface. In an embodiment 
that is very efficient as far as calculation is concerned 
but is relatively inaccurate, the imaging relationship is 
derived by finding which image transform gives rise to 
the required relationship between the candidate vectors, 

10 whereby rotation and perspective in the image are mini- 
mised on the basis of this imaging relationship. 

According to an alternative embodiment, the main 
vectors are selected instead on the basis of earlier 
main vectors which were determined for a previous image. 

15 It is possible to make the main vectors identical to the 
earlier main vectors, or to use weighted information 
about these earlier main vectors during the identifica- 
tion of the main vectors, for example in order to make 
the localisation of peaks in the spatial frequency spec- 

2 0 trum more effective. 

According to a preferred embodiment, the marks are 
transformed with the identified main vectors as the bases 
for the production of a rotation- corrected image in which 
rotation of the marks over the plane of the image is 

25 essentially eliminated. 

According to a further preferred embodiment, a com- 
pensation for the perspective is effected in the thus 
rotation-corrected image. This compensation can be pre- 
ceded by a needs test, in which the width is determined 

30 of the peaks corresponding to the main vectors in a spa- 
tial frequency spectrum of said rotation-corrected image. 
If the width exceeds a given width value, perspective 
compensation is carried out. The width of the peaks can 
be determined in a way that is efficient as far as cal- 

35 culation is concerned, since the positions of the peaks 
in the spatial frequency spectrum are essentially known 
from preceding processing steps. 
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The perspective compensation suitably comprises mea- 
suring an inclination variation for the raster pattern 
along each main vector in the rotation-corrected image, 
calculating a perspective transform that essentially eli- 
5 minates the inclination variation on the basis of the 
measured inclination variation, and producing a perspec- 
tive-corrected image by means of the perspective trans- 
form. This is an efficient way to calculate a perspec- 
tive-corrected image with a high degree of accuracy. 

10 In this connection, it is preferable that the mea- 

surement of the inclination variation for the raster pat- 
tern along a selected main vector comprises, via Fourier 
analysis of at least two subsets of the rotation-correct- 
ed image distributed along the selected main vector, cal- 

15 culating at least one subset main vector for each subset, 
identifying an initial position in the associated subset 
for each subset main vector, and calculating the inclina- 
tion variation along the selected main vector on the 
basis of the subset main vectors and the initial posi- 

20 tions. The identification of the subset main vectors via 
Fourier analysis can be achieved quickly by starting from 
the known main vectors. In general, the subset main vec- 
tors are situated close to the main vectors. 

The initial position is preferably identified on the 

25 basis of the centre of gravity of the marks incorporated 
in the respective subset. This means that the identifica- 
tion of the initial position is only affected to a small 
extent by imaging errors, such as high degree of perspec- 
tive, missing marks, additional marks, noise, etc. Alter- 

30 natively, the initial position could be set as the mid- 
point of the subset, whereby the calculations required 
are minimised. 

A further preferred embodiment comprises the steps 
of measuring the phase displacement along the respective 

35 main vector via Fourier analysis of the rotation-correct- 
ed or perspective-corrected image, and on the basis of 
the measured phase displacements localising the raster 
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pattern relative to the marks in the image. The phase 
displacement is suitably obtained as the phase angle for 
the two-dimensional Fourier transform of the image for 
the main vectors and can easily be eliminated by a trans- 
formation operation. The directions of the raster lines 
are then given by the perpendiculars to the main vectors, 
and the distance between the intersections of the raster 
lines along the main vectors is given by the lengths of 
the main vectors. 

In certain cases, it is desirable to calculate a 
normalising transform that places the intersections of 
the raster pattern a given distance apart, for example, 
at integer coordinates, and to operate the normalising 
transform on the image in order to produce a normalised 
15 image. 

According to another aspect of the present inven- 
tion, this relates to a computer-readable computer pro- 
gram product which comprises a computer program with 
instructions for causing the computer to implement the 

20 method for identification as described above. This com- 
puter program product can, for example, comprise a non- 
volatile memory for a computer, such as a floppy disk or 
CD ROM, or a volatile memory in a computer. The computer 
program product can alternatively comprise propagating 

25 signals, such as a bit stream for packet transfer via the 
Internet or the like, or carrier waves that are transmit- 
ted to the computer by cable-based or wireless means. 

According to a further aspect of the present inven- 
tion, this relates to a device for position determina- 

3 0 tion. The device comprises a sensor for producing an 

image of a partial surface of a surface which is provided 
with a position code in the form of a plurality of marks, 
each of which is associated with one of a plurality of 
intersections belonging to a virtual raster pattern, and 

35 an image-processing means which is arranged to calculate 
a position for the partial surface based on a subset of 
the surface. The image-processing means is thereby 
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designed to identify the virtual raster pattern in accor- 
dance with the method above. 

The advantages of the computer program product and 
the device for position determination are apparent from 
5 the above description. The features described in asso- 
ciation with the method for identifying a virtual raster 
pattern are, of course, also applicable to the device for 
position determination. 
Brief Description of the Drawings 

In the following, the invention will be described 
10 for the purpose of exemplification with reference to the 
accompanying drawings which show a currently preferred 
embodiment and in which 

Figs 1A-1B show an example of a surface with a 
position- coding pattern and an image of the same respec- 
15 tively, 

Fig. 2 illustrates some of the processing steps that 
are implemented for identification of the virtual raster 
pattern in the image according to Fig. IB, 

Fig. 3 shows the two-dimensional Fourier transform 
2 0 of the image according to Fig. IB, with the actual calcu- 
lation dots marked by black symbols, 

Fig. 4 shows the image according to Fig. IB after 
rotation compensation, 

Figs 5A-5D show two-dimensional Fourier transforms 

2 5 of subsets of the image according to Fig. 4, with the 

actual calculation dots marked by black symbols, 

Fig. 6 is a diagram of the calculated inclination 
variation along two main vectors in the image according 
to Fig. 4, 

3 0 Fig. 7 shows the image according to Fig. 4 after 

elimination of perspective, 

Fig. 8 shows a two-dimensional Fourier transform of 
the image according to Fig. 7, with the actual calcula- 
tion dots marked by symbols, 
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Fig. 9 shows the image according to Fig. 7 after 
compensation for displacement along the main vectors, the 
original orthogonal raster pattern being identified, 

Fig. 10 shows a device that can be used for position 
5 determination, 

Fig. 11 shows a position-coding pattern with a tri- 
angular raster pattern, and 

Fig. 12 shows a position-coding pattern with a hexa- 
gonal raster pattern. 

Description of Preferred Embodiments 

10 Fig. la shows a part of a product 1 which on at 

least part of its surface 2 is provided with an optical- 
ly readable position-coding pattern 3 which makes pos- 
sible position determination. The position-coding pat- 
tern 3 comprises marks 4 which are systematically arrang- 

15 ed across the surface, so that this has a "patterned" 

appearance. The position determination can be carried out 
on the whole surface of the product. In other cases, the 
surface that permits position determination can consti- 
tute a smaller part of the product. The product can, for 

2 0 example, be used to produce an electronic representation 

of information that is written or drawn on the surface. 
The electronic representation can be produced by deter- 
mining the position of a pen on the paper continually 
while writing on the surface with the pen, by reading off 
25 the position-coding pattern. 

More particularly, the position-coding pattern com- 
prises a virtual raster 5 (indicated by broken lines in 
Fig. 9), which is neither visible to the human eye nor 
can be detected directly by a device for determining 

3 0 positions on the surface, and the marks 4, each of which 

represents one of four values dependent upon its loca- 
tion. The value of a mark 4 depends upon its location in 
relation to its nominal position 6 (Fig. 9) , which can 
also be called its virtual raster point and is repre- 
35 sented by the intersection between the raster lines 7 

(Fig. 9) . The mark 4 is the shape of a circular dot. In 
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the example in Fig. 9 there are four possible locations, 
one on each of the raster lines 7 extending from the 
nominal position 6. The displacement from the nominal 
position 6 is the same size for all values. In this case, 
5 the virtual raster 5 is orthogonal and has the same pitch 
in both its main directions or raster directions. In this 
connection it should be pointed out that the position- 
coding pattern in Fig. 1A and Fig. 9 have been greatly 
enlarged for the sake of clarity. In addition, it is 

10 shown on only a part of the product. 

For details concerning the generation of the posi- 
tion-coding pattern and the decoding of the same for 
position determination, reference is made to the Appli- 
cant's Swedish Patent Application PCT/SE00/01895 . 

15 in order for the position code to be able to be 

detected, the virtual raster needs to be determined. This 
must be carried out with high accuracy and in real time, 
based on an image of a partial surface with a number of 
marks 4. Fig. IB shows such an image of the partial sur- 

20 face 8 in Fig. 1A. For the sake of clarity, the imaged 
marks are somewhat larger than the other marks in 
Fig. 1A. In the image according to Fig. IB the pattern is 
distorted due to the sensor not being held parallel with 
the imaged partial surface. The image therefore contains 

2 5 both an unknown perspective, due to tilting of the sensor 
relative to the partial surface, and an unknown rotation 
in the image plane, due to rotating or skewing of the 
sensor around its perpendicular axis. 

Fig. 2 indicates the main steps for the recreation 

30 of the original dot pattern and identification of the 

virtual raster based on a recorded image. The main steps 
comprise preprocessing of the image (step 100) , detection 
of the image's main vectors via Fourier analysis (step 

101) , compensation for rotation in the image plane by 
35 projection of the marks along the main vectors (step 

102) , detection of the perspective in the rotation- 
corrected image via Fourier analysis (step 103) , removal 
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of the detected perspective by transformation (step 104) , 
detection of the phase content in the perspective-cor- 
rected image via Fourier analysis (step 105) and finally 
removal of the phase content by transformation to produce 
5 a recreated dot pattern with the associated virtual 
raster (step 106) . 

In the following, each of the main steps will be 
described in greater detail. 
Preprocessing (Step 100) 



10 the marks 4 in the image. This can be carried out using 
thresholds, so that a mark is identified by one or more 
picture elements (pixels) with a value that exceeds a 
predetermined or calculated threshold value. The centre 
of gravity is calculated for all such picture elements 

15 belonging to a mark and is used in the subsequent pro- 
cessing. Thus, the appearance of the mark in the image 
has as little effect as possible on the mark's subse- 
quently calculated displacement from the nominal posi- 
tion. The image is then converted to a set of dots by 

20 the marks being replaced by unit pulses (so-called Dirac 
pulses, 5) which are placed at the centres of gravity of 
the marks . 

When the set of dots has been defined, a number of 
partial steps are implemented that are based on Fourier 
2 5 analysis of the image, or rather the above-mentioned set 
of dots. 

The general two-dimensional Fourier transform of an 
image f (x,y) is of the form: 



The aim of the preprocessing is to identify all 




image 



30 



Since the image is converted to a set of dots, the 
Fourier transform is given by the expression: 
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F(u,v)= //flx-x^y-y^e-^"-*^ 

image 

_ ^^-2-7i i (u xj+y yj c ) 
• j,k 

where the set of dots is {x-j, y k }, and (u,v) is a direc- 
tion vector, since the integral of a Dirac pulse is one 
(1), and the image function between the Dirac pulses is 
5 zero (0) . 

This expression can be calculated relatively quick- 
ly, as the number of operations is only equal to the num- 
ber of detected dots, that is normally of the order of 
100. 

Detection of Main Vectors (Step 101) 

10 After the preprocessing of the image, the set of 

dots is analysed to detect its main or raster vectors, 
that is its dominant directions and dominant spatial 
frequencies. This is carried out by the calculation of 
the Fourier transform of the set of dots for different 

15 direction vectors (u,v) . The absolute values of the cal- 
culated Fourier transforms |F(u,v) | give a spatial fre- 
quency spectrum in two dimensions. 

Fig. 3 shows a spatial frequency spectrum for the 
set of dots in Fig. lb. Dark areas indicate higher spec- 

20 tral amplitudes. The central dark area forms the origin, 
that is (u,v) = (0,0). Instead of calculating the whole 
two-dimensional frequency spectrum as shown in Fig. 3, 
the Fourier transform is calculated at a number of dots, 
that is for a number of different direction vectors 

25 (u,v) . These dots are marked in Fig. 3. More particular- 
ly, a band- shaped area in a half plane of the frequency 
domain is "swept" in steps. Several sums are calculated 
along one and the same search direction. With the aim of 
making the calculations more efficient, only scaling is 

30 carried out along the search direction, and the scalar 
product (u 0 -Xj+vo -yjj is calculated only once per direc- 
tion vector (u 0 ,v 0 ) . The scaling is carried out itera- 
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tively and gives rise to only one extra addition per step 
and dot in each search direction. 

In pseudo-code this can be written: 

for dot=l to N 
5 a0=u0-dot.x+v0- dot.y 

da0=a0/8 
for s=0 to 11 

dsum=exp ( -2- n- i • aO) 
sum(s) = sum(s) + dsum 
10 a0=a0+da0 

next s 
next dot 

In each search direction, steps can suitably be used 
that are inversely proportional to any suitable power of 
15 2 (2 n ) , for example, eighths as above, as the calculation 
of the step (da 0 ) is thereby reduced to a bit shift that 
is efficient as far calculation is nonr^rned. 

The width of the search band in the frequency plane 
is determined based on how much the imaging distortion 

2 0 can be assumed to change the spatial frequencies of the 

original pattern (Fig. 1A) along the main vectors. This 
depends in turn upon in what locations the imaging sensor 
can be expected to be placed. Normally the search band 
extends on both sides of the pattern's nominal spatial 
25 frequencies, that is the distance between the raster 
lines in the original pattern. Of course, the size of 
the steps in each search direction and the steps in the 
sweep direction varies depending upon available processor 
power, required calculation time and required accuracy. 

3 0 From the above, it can be seen that the searching in 

the frequency plane is carried out by the calculation of 
a number of independent sums. The searching is thus of a 
parallel nature and is therefore well suited for imple- 
mentation in hardware, such as an ASIC. 
35 After or during the calculation of this "sampled" 

spatial frequency spectrum in two dimensions, candidate 
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vectors cl-c3 are identified, based on the values of the 
calculation dots. All calculation dots with a value above 
a threshold value are considered to be significant (mark- 
ed with black crosses in Fig. 3) and are used for the 
5 calculation of the coordinates for the candidate vectors 
cl-c3. The threshold value is determined as a percentage, 
for example 50%, of the theoretically maximum amplitude 
value, which is proportional to the number of marks in 
the image. The number of marks is known from the prepro- 

10 cessing step. If several significant calculation dots are 
adjacent to each other in the frequency plane, the centre 
of gravity of these is considered to give the coordinate 
for a candidate vector cl-c3 in the frequency plane (can- 
didate vectors which later prove to be main vectors or 

15 diagonal vectors are marked by white and black stars 
respectively in Fig. 3) . 

It must be pointed out that all dots in the set of 
dots are not used for the above calculation, but only 
those that are closest to the centre of the image. Typi- 

20 cally, approximately 50% of the available dots are used. 
This is done principally for two reasons. If all the dots 
were to be included, there is a danger that the peaks in 
the frequency plane (Fig. 3) would become too wide, and 
thereby difficult to detect, particularly with a high 

25 degree of perspective. This is because the perspective 

gives rise to great frequency changes in peripheral parts 
of the image. At the centre of the image, however, the 
spatial frequencies correspond relatively well to the 
spatial average frequencies in the image. In addition, 

30 the problem is avoided of insufficient illumination of 

the periphery of the image, and the resulting high noise 
levels. In addition, the calculation time is reduced in 
proportion to the reduction in the number of dots. 

After the above calculation step, a number of peaks 

3 5 have been detected in the frequency plane. However, it 

remains to be determined which of the detected peaks cor- 
respond to main vectors. If the image is recorded without 
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perspective between the sensor and the patterned product, 
the diagonal vector can be distinguished in that it has 
a length that is 4l times longer than the main vectors, 
which in addition are orthogonal to each other. This does 
5 not apply, however, with more extreme perspectives, such 
as in Fig. IB, where the main vectors are not orthogonal, 
and neither is the length ratio between the diagonal 
vector and the main vector In addition, one or more 

invalid maximums may have been detected in the frequency 

10 plane, for example as a result of interference or noise 
in the image. 

For the identification of main vectors a number of 
image transforms are used, typically approximately 25, 
corresponding to a number of given imaging relationships 

15 between the sensor and the product. In the case of a pen, 
this can in fact be physically angled and rotated rela- 
tive to the product in a limited number of ways. The 
image transforms restore a given distorted frequency 
domain to an orthogonal and uniform frequency domain, 

20 that is a frequency domain where the main vectors actual- 
ly are orthogonal and where the length ratio between a 
diagonal vector and a main vector is The image 

transforms typically give rise to a so-called ellipti- 
cal transformation. 

25 After identification of the candidate vectors cl-c3, 

these candidate vectors cl-c3 are operated on by the 
image transforms and it is measured how well the proper- 
ties (direction and length) of the transformed candidate 
vectors appear to correspond to those of the true main 

3 0 vectors. 

The above can be implemented by having all the image 
transforms operate on all candidate vectors cl-c3, by the 
transformed candidate vectors being allocated ratings 
based on their mutual relationships, and by the main 
35 vectors being identified as the transformed candidate 

vectors that received the highest ratings. If necessary, 
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the image transform that gave rise to the highest ratings 
can also be identified. 

An alternative which is more efficient as far as 
calculation is concerned is to divide the calculation 
5 into two partial steps. In the first partial step, a 
sequence of image transforms may operate on all pairs 
of candidate vectors cl-c3, in order to identify all 
potential pairs of main vectors and the image transforms 
that gave rise to these potential pairs. After transfor- 

10 mation, potential pairs of main vectors have an orthogo- 
nal mutual angle ratio, a mutual length ratio of 1:1 and 
given lengths. These criteria are of course determined 
within given tolerances. The criterion that the vectors 
must have given lengths after transformation is intended 

15 to exclude two mutually orthogonal diagonal vectors being 
mistaken for main vectors. In this first partial step, 
the operation of different image transforms on a given 
pair of candidate vectors is suitably discontinued when 
this given pair is identified as a pair of potential main 

2 0 vectors, whereupon the sequence of image transforms may 
operate on a new pair of candidate vectors. This reduces 
the number of operations required. In the second partial 
step, the image transforms that gave rise to the poten- 
tial pairs may operate on any additional candidate vec- 

2 5 tors, whereupon these additional candidate vectors are 

allocated ratings in accordance with their correspondence 
with diagonal vectors. The main vectors can then be iden- 
tified as the pair of candidate vectors that received the 
highest ratings. The division into two partial steps 
30 means that the sometimes relatively calculation-intensive 
ratings evaluation only needs to be implemented for a 
small number of pairs of candidate vectors and a small 
number of image transforms. 

According to a further alternative, a sequence of 

3 5 image transforms may operate on all pairs of candidate 

vectors. If sufficiently high ratings are obtained for a 
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20 

particular image transform, then this is selected, other- 
wise the next image transform in the sequence is tested. 

In this connection, it is possible to test the image 
transforms adaptively, that is for example on the basis 
of the user's identity and/or the image transform that 
was selected for a previous image. It is, in fact, prob- 
able that consecutive images have been imaged under simi- 
lar conditions. One and the same user also probably car- 
ries out the imaging in a similar way each time. 

It is worth noting that the image transforms and the 
associated calculations are simple to carry out as only 
the actual candidate vectors are transformed, and not 
the whole set of dots. The image transforms can be 2x2 
matrices, and the number of operations is thus four per 
15 image transform and pair of candidate vectors. 

As each image transform corresponds to a known rela- 
tionship between the sensor and the product, the above 
detection step provides an indication of how the sensor 
is held relative to the product. With low accuracy 
20 requirements, this information can be used immediately 
in order to compensate for perspective and rotation in 
the image. Often, however, a more accurate compensation 
is required. 

According to an alternative procedure, searching for 
25 peaks in the frequency plane is not carried out. Instead, 
an image transform is selected initially that is operated 
on the set of dots or a subset thereof. Thereafter, the 
main vectors of the transformed set of dots are identi- 
fied via Fourier analysis, suitably by the above -ment ion - 
3 0 ed searching for peaks in the frequency plane. If no 

satisfactory result is obtained, a new image transform is 
selected, whereupon a new identification of main vectors 
is implemented via Fourier analysis. This method is, how- 
ever, relatively calculation-intensive, particularly if 
35 the image conditions change frequently between consecu- 
tive images. To minimise the number of image transforms 
that must be tested, the selection of image transform is 
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suitably based on the user's identity and/or the image 
transform selected for a previous image. 
Compensation for Rotation (Step 102) 

In order to compensate for rotation of the set of 
dots in the image plane, the set of dots is projected 
5 along the main vectors detected as described above. A set 
of dots is thereby obtained which is principally aligned, 
even though there can still remain a non-linear interfe- 
rence in the form of a perspective in the image. 

Fig. 4 shows the set of dots in Fig. IB after such 

10 an alignment. In order to show clearly the remaining per- 
spective, the corresponding virtual raster has also been 
marked by broken lines. It should be noted that this 
raster has been calculated based on parameters that are 
determined in the immediately-following detection of per- 

15 spective (step 103) . 

Detection of Perspective and Comp ensation for this 
(Steps 103-104) 

A general characteristic of perspective is that ori- 
ginally straight lines are imaged on lines that converge 
towards a vanishing point. It can be shown, see Appendix 
A, that these lines intersect coordinate axes (x a , y s ) of 

20 the sensor (the image) in such a way that the inclina- 
tions of the lines (Ax s /Ay s , Ay s /Ax s ) increase linearly 
along the respective coordinate axis: 

^^- = kk x x 8 +mk x 

T^ = kk y -Ys + ^yy 
Ax s y 

Appendix A also shows that an inclined plane that 
25 gives rise to this perspective has a z-coordinate that 
varies according to the formula: 

z = C - kky • x - kk x • y , 
where x, y are spatial coordinates on the inclined plane, 
that is on the surface of the product, and C is a scaling 
30 factor. 
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In addition, Appendix A shows that perspective can 
be compensated for by means of the perspective transform: 




1 - kk x • Xg - kk y • yg 



I - kk x ■ Xg - kk y • yl 



where 

5 x|=x s -y s mk x 
Ys =y s -x s mk y 

Before this transformation can be carried out, it 
is thus necessary to measure how the inclination of the 
raster pattern varies along the main vectors of the rota- 
tion-corrected set of dots (corresponding to the coordi- 

10 nate axes x S/ y s above). This inclination variation gives 
the desired values of mk x/ mk y/ kk x/ kk y . 

The inclination variation is measured via Fourier 
analysis. The set of dots is divided into four subsets 
around the image's/sensor's horizontal and vertical sym- 

15 metry axes, the directions of which in practice essen- 
tially coincide with the directions of the main vectors. 
Each subset comprises dots in a half plane: above the 
horizontal symmetry axis, below the horizontal symmetry 
axis, to the right of the vertical symmetry axis, to the 

20 left of the vertical symmetry axis. Unlike in the above- 
described detection of main vectors (step 101) , the whole 
set of dots is used here for the division into subsets. 
The Fourier transform for the respective subsets is cal- 
culated in a corresponding way as for the detection of 

25 main vectors. The sweeping of the frequency plane is car- 
ried out around the main vector and continues until all 
significant, adjacent peak values have been detected. In 
a corresponding way as for the detection of the main vec- 
tors, the position of the peak is calculated as the cen- 

30 tre of gravity of all the calculation dots that are adja- 
cent to each other in the frequency plane. 
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Figs 5A-5D show the calculation for the respective 
half planes: upper (Fig. 5A) , lower (Fig. 5B) , left 
(Fig. 5C) , right (Fig. 5D) , wherein black dots indicate 
calculation dots without detection of any significant 
5 peak value, black crosses indicate calculation dots with 
detection of a significant peak value, and white stars 
indicate the centres of gravity for the respective peaks. 
These centres of gravity define subset main vectors, the 
direction of which gives the inclination (Ax s /Ay s , 

10 Ay s /Ax s ) of the set of dots in the respective half plane. 
It should be pointed out that a complete frequency spec- 
trum in two dimensions is not calculated. For the sake 
of clarity it is, however, shown in the background of 
Figs 5A-5D, wherein the central dark area forms the ori- 

15 gin, that is (u,v)= (0,0). 

For the determination of mk x , mk y , kk x , kk y , initial 
positions are also calculated for the subset main vectors 
in the half planes. The initial positions are calculated 
suitably as the centre of gravity of the set of unit 

20 pulses in the respective half plane, since the centre of 
gravity is relatively insensitive to variations in the 
sensor's position, lost dots and a high degree of per- 
spective. It is, however, possible instead to locate the 
initial position at the geometric centre of the respec- 

25 tive half plane. For the subset main vectors associated 
with the left and right half planes, the intersections of 
the subset main vectors with the horizontal symmetry axis 
are calculated, and for the subset main vectors associat- 
ed with the upper and lower half planes, the intersec- 

30 tions of the subset main vectors with the vertical sym- 
metry axis are calculated. The inclinations and inter- 
sections are fitted to a straight line, from which mk x , 
mk y , kk x , kk y are calculated. Fig. 6 shows a diagram of 
the inclination along a symmetry axis as a function of 

35 the position on this symmetry axis. Black crosses indi- 
cate the measurement dots that were obtained by the 
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Fourier analysis as above based on the set of dots in 
Fig. 4. 

When mk x/ mk y , kk x/ kk y are known, a compensation for 



5 above. Fig. 7 shows the set of dots in Fig. 4 after com- 
pensation for perspective. The corresponding virtual 
raster has also been marked by broken lines. 

Of course, it is possible to divide the set of dots 
into other subsets than those described above for calcu- 
10 lation of the inclination variation. More subsets than 
two per main vector can be used for increased accuracy. 

It should also be pointed out that other transforms 
can be used to compensate for perspective. For example, 
the above perspective transform can be replaced by the 
15 approximating transform: 



The inclination variation of the rotation-compensat- 
ed image can be measured by other methods than Fourier 
analysis, for example by line fitting according to the 

20 least-squares method. Unlike the Fourier-based method 
described, which handles the set of dots as a single 
entity, such line fitting methods require local decisions 
concerning individual dots, for which reason they are 
more sensitive to interference. 

2 5 Detection of Displacement and Compensation f or this 
(Steps 105-106) 

After the compensation for perspective, there 
remains in principle only a constant displacement along 
the main vectors. As shown in Fig. 7, the raster is 

30 otherwise principally straight and essentially free of 
rotation in the image plane. The displacements can be 
measured as the phase in the Fourier analysis. What 
remains is thus to measure this phase and to compensate 
for it . 



perspective is implemented via the perspective transform 
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The Fourier transform for the whole set of dots is 
calculated in a corresponding way to that for the detec- 
tion of main vectors. The sweeping of the frequency plane 
is carried out around the main vectors that were identi- 
5 fied in step 101 above and continues until all signifi- 
cant adjacent peak values have been detected. In a cor- 
responding way to that for the detection of the main 
vectors (step 101) , the position of the peak is calcu- 
lated as the centre of gravity of all the calculation 

10 dots that are adjacent to each other in the frequency 

plane. The result for the set of dots in Fig. 7 is shown 
in Fig. 8, wherein again black dots indicate calculation 
dots without detection of any significant peak value, 
black crosses indicate calculation dots with detection 

15 of a significant peak value, and white stars indicate 

the centre of gravity of the respective peaks. It should 
be pointed out that a complete two-dimensional frequency 
spectrum is not calculated. For the sake of clarity it 
is, however, shown in the background of Fig. 8, where - 

2 0 in the central dark area forms the origin, that is 

(u,v) = (0,0). Fig. 8 also shows that the peaks in the 
frequency plane become narrower after compensation for 
perspective. The width of the peaks can thus be used to 
determine whether there is in fact a need for perspective 

2 5 compensation in an image. 

At the same time as localisation of the respective 
main vector via the absolute value of the Fourier trans- 
form, the phase displacement along the respective main 
vector from the Fourier transform's phase angle is 

30 obtained, which is given by the ratio between its real 
part and its imaginary part. This phase displacement is 
eliminated by phase transformation of the set of dots, 
whereupon a new projection is carried out along the most 
recently determined main vectors. In addition, there is 

35 scaling, on the basis of the length of the main vectors, 
to ensure that the raster is based on integer coordi- 
nates . 



WO 01/75783 PCT/SE01/00604 




26 

Fig. 9 shows the re-created image after phase com- 
pensation. As shown, the virtual raster 5 is correctly 
localised relative to the marks/unit pulses, which can 
now be decoded for position information. 
5 According to an alternative, there is no searching 

for new main vectors in the frequency plane, since after 
steps 101-102 these are relatively well-defined. Instead, 
the phase angles are used for the Fourier transforms 
which are calculated for the main vectors for the detec- 

10 tion of these (step 101) . 

Optionally, a supplementary fine adjustment of the 
identified raster pattern can be carried out after the 
above-mentioned phase compensation. As each mark is a 
known distance from its nominal position, each sequence 

15 of marks along each main vector can be fitted, for exam- 
ple via a least-squares method, to a line, which forms a 
fine-adjusted raster line. 
Device for Position Determination 

An embodiment of a device for position determination 
is shown schematically in Fig. 10. It comprises a casing 

20 11 which has approximately the same shape as a pen. In 
the short side of the casing there is an opening 12. The 
short side is intended to abut against or to be held a 
short distance from the surface on which the position 
determination is to be carried out. 

25 The casing contains principally an optics part, an 

electronic circuitry part and a power supply. 

The optics part comprises at least one light-emit- 
ting diode 13 for illuminating the surface which is to be 
imaged and a light-sensitive area sensor 14, for example 

30 a CCD or CMOS sensor, for recording a two-dimensional 

image. Optionally, the device can also contain an optical 
system, such as a mirror and/or lens system. The light- 
emitting diode can be an infrared diode and the sensor 
can be sensitive to infrared light. 
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The power supply for the device is obtained from a 
battery 15, which is mounted in a separate compartment 
in the casing. 

The electronic circuitry part contains an image- 
5 processing means 16 for determining a position on the 

basis of the image recorded by the sensor 14 and in par- 
ticular a processor unit with a processor which is pro- 
grammed to read images from the sensor and to carry out 
position determination on the basis of these images. 

10 In this embodiment, the device also comprises a pen 

point 17, with the aid of which ordinary pigment -based 
writing can be written on the surface on which the posi- 
tion determination is to be carried out. The pen point 17 
can be extendable and retractable so that the user can 

15 control whether or not it is to be used. In certain 

applications the device does not need to have a pen point 
at all. 

The pigment-based writing is suitably of a type that 
is transparent to infrared light and the marks suitably 

20 absorb infrared light. By using a light-emitting diode 
which emits infrared light and a sensor which is sensi- 
tive to infrared light, the detection of the pattern can 
be carried out without the above-mentioned writing inter- 
fering with the pattern. 

2 5 The device can also comprise buttons 18, by means of 

which the device can be activated and controlled. It has 
also a transceiver 19 for wireless transmission, for 
example using infrared light, radio waves or ultrasound, 
of information to and from the device. The device can 

30 also comprise a display 20 for displaying positions or 
recorded information. 

A device for recording text is described in the 
Applicant's Swedish Patent No. 9604008-4. This device can 
be used for position determination if it is programmed in 

35 a suitable way. If it is to be used for pigment-based 
writing, then it must also be given a pen point. 
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The device can be divided between different physical 
casings, a first casing containing components which are 
required for recording images of the position-coding pat- 
tern and for transmitting these to components which are 
5 contained in a second casing and which carry out the 

position determination on the basis of the recorded image 
or images . 

As mentioned, the position determination is carried 
out by a processor which thus must have software for 

10 locating marks in an image and decoding them and for 

determining positions from the codes thus obtained. Based 
on the example above, a person skilled in the art will be 
able to design software which carries out the above-men- 
tioned identification of the virtual raster pattern on 

15 the basis of an image of a part of a position-coding pat- 
tern. 

The device is preferably specified to be used with 
imaging conditions that lie within given limits. The 
imaging conditions can be defined as a permitted incli- 

20 nation or tilting of the device (the sensor 14) relative 
to the surface that is to be imaged, for example a maxi- 
mum of approximately 60° , and also a permitted rotation 
or skewing of the device around its longitudinal axis, 
for example in a range approximately ±3 0° relative to a 

25 reference position. 

In the embodiment above, the pattern is optically 
readable and the sensor is therefore optical. The pattern 
can, however, be based on a parameter other than an opti- 
cal parameter. In such a case, the sensor must of course 

3 0 be of a type which can read off the parameter concerned. 
Examples of such parameters are chemical, acoustic or 
electromagnetic marks. Capacitive or inductive marks can 
also be used. 

It is recognised that many variations are possible 
35 within the scope of the present invention. The order of 

the above steps 101-106 can be varied within the scope of 
the invention. For example, detection of and compensation 
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for displacement can be carried out before detection of 
and compensation for rotation and/or perspective. It is, 
however, preferable to carry out the displacement steps 
after the rotation and perspective processing steps, 
5 since the displacement steps then become particularly 
simple as far as calculations are concerned. 

It is also possible to carry out the perspective 
processing steps before the rotation processing steps, 
suitably after having first converted the image into a 

10 set of dots. Such a step for detection of perspective 

can, for example, comprise the division of the image into 
a plurality of subsets, and detection of at least one 
direction in each subset, for example via Fourier ana- 
lysis of the respective subsets. After this the change in 

15 the detected directions across the image is evaluated for 
the calculation of a transform that compensates for the 
perspective in the image. The same transform can possibly 
also compensate for rotation in the image plane. The eva- 
luation can, for example, result in an identification of 

20 the image's vanishing points, which are then transformed 
into positions in infinity on the symmetry axes of the 
image, for example by means of a perspective transform 
of the type used in step 104 above. If the transformation 
only compensates for perspective, this is suitably fol- 

25 lowed by detection of rotation and compensation for this, 
for example in accordance with the above steps 101-102. 

In addition, the marks can have a different appear- 
ance than described in the example above. Each mark can, 
for example, consist of a line or an ellipse, which 

3 0 starts at the virtual raster point and extends from this 
to a particular position. Alternatively, some other sym- 
bol can be used, such as a square, rectangle, triangle, 
circle or ellipse, filled-in or not. 



35 raster lines in an orthogonal raster but can also have 
other arrangements, such as along the raster lines in 
a raster with 60 degree angles, etc. 



Nor do the marks need to be arranged along the 
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Rasters in the form of triangles or hexagons can 
also be used, as shown in Figs 11 and 12. For example, 
a raster with triangles, see Fig. 11, enables each mark 
to be displaced in six different directions. In a hexa- 
5 gonal raster (honeycomb raster), see Fig. 12, the marks 
can be displaced in three different directions along 
the raster lines. In these cases the main vectors of 
the image are correspondingly more in number; in 
Fig. 11 there are three dominant raster directions, and 

10 in Fig. 12 there are three dominant raster directions. 

As mentioned, the marks do not need to be displac- 
ed along the raster lines but can be displaced in other 
directions, for example in order to be located each in 
a separate quadrant of a square raster pattern. In the 

15 hexagonal raster pattern, the marks can be displaced in 
four or more different directions, for example in six 
different directions along the raster lines and along 
lines which are at 60 degrees to the raster lines. In an 
orthogonal raster, if necessary, only two displacements 

20 can be used. 

It must also be pointed out that depending upon the 
available processing power, it can be possible to carry 
out the above-mentioned Fourier analysis based on all the 
actual image information that is recorded by the sensor. 

25 Similarly it can, if the processor power is sufficient, 
be possible to calculate a complete spatial frequency 
spectrum in two dimensions, as shown in the background 
to Figs 3, 5 and 8. 

Those skilled in the art will also recognise that 

30 the analysis in the frequency domain described above can 
be implemented in a corresponding way in the wavelength 
domain. 
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APPENDIX A 

An optical system with amplification m images the 
spatial coordinate (x,y,z) on the sensor coordinate 
(x s ,y s ) according to the projection formula: 



10 



Ys = 



x 

-m — 
z 

m Y 

— m — 



(1) 



Assume now two points in the space, P 1 and P 2 . These 
lie on an inclined plane, the z-component of which is 
described by z=z 0 +ax+by. The points P 1 , P 2 lie symmetri- 
cally on each side of the x-axis, in the position x 0/ and 
have the coordinates: 

Jp 1 = (x 0 ,y 0 , z 0 + ax 0 + by 0 ) 
|p> = (x 0 ,-y 0 , z 0 + ax 0 - by 0 ) 

After projection in accordance with (1) the points 
P 1 , P 2 end up on the sensor coordinates P s x , Ps 2 : 
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z 0 + ax 0 + by c 
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15 



We now calculate the inclination in the sensor, 
Ax s /Ay s , between P s l and P s 2 : 
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(2) 



A point P 3 in the centre of the x-axis, in position 
x 0 , P 3 = (x 0 ,0,z 0 + ax 0 ) , is imaged in accordance with (1) 
by: 
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f ^ .0] , (x 8 ,o) 



(3) 



If we compare (3) with (2), we see that the incli- 
nation in the sensor is changed linearly in accordance 
with: 



x B b 
— — x. 

y 8 m 



(4) 



For reasons of symmetry, a corresponding equation 
applies to the inclination in the y-direction. Thus we 
now know: 



10 



15 



20 



m 



Ax c 



(y B ) = - Ys 



m 



A. raster situated on the inclined plane will, in 
addition to this strictly linear component, also con- 
tain a constant inclination, if the raster is not per- 
fectly centred. We then get the following more general 
equation: 



^.(x s ) = -x s + mk x = kk, 
Ay s tn 

Ay s / \ a 



Ax 6 



(y s ) = -y« + mk y 3 kk > 

m 



x s + mk x 



y s + mk 



(5) 



A general perspective can thus be described by the 
four variables mkx, mky, kkx, kky. It now remains to find 
the transform that restores the image coordinates to an 
orthogonal condition, given mkx, mky, kkx, kky. In order 
to do this, we first subtract the constant part of the 
inclination (originating from the remaining rotation) so 
that the inclination at the coordinate axes is zero: 



ixl = x s - y s • mk,, 



(6) 
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(1) is then used to reverse- transform these rota- 
tion-corrected coordinates. It is possible without the 
loss of generality to put z0=m, as this only results in 
a scaling of the coordinates. We have 



from (1) 



x = 



y = 



z • x~ 

- m 

z ; Ys 

- m 



from (5) : f a - m ' kk y 
lb = m • kk x 



and the assumption: z = m+ax+by, 

which gives us the non-linear equation system: 

(m + m • kk x • x + m ■ kk y • y) • Xg 
- m 

(m + m • kk x • x + m • kk y ■ y) • y* 



x = 



y - 



- m 



10 whose solution is 



y = - 



1 - kk x • x; - kk y • yl 

yI 

1 - kk x • - kk y • yl 



where according to (6) 
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CLAIMS 

1. A method for identifying a virtual raster pat- 
5 tern (5) in an image of a plurality of marks (4) , each 

of which is associated with a respective intersection 
(6) of raster lines (7) belonging to said raster pattern 
(5), characterised in that the virtual raster 
pattern (5) is identified via Fourier analysis of said 
10 image. 

2. A method according to claim 1, comprising the 
step of converting the image to a set of unit pulses 
before the Fourier analysis, which unit pulses are plac- 
ed at the positions of the marks (4) in the image. 

15 3. A method according to claim 2, wherein each unit 

pulse is placed at the centre of gravity of the corre- 
sponding mark (4) . 

4. A method according to any one of claims 1-3, 
wherein said Fourier analysis comprises the steps of 

20 calculating a spatial frequency spectrum in two 

dimensions on the basis of said image, 

identifying at least two main vectors in said image, 
based on said frequency spectrum, and 

identifying the raster lines (7) in said raster pat- 
25 tern (5), on the basis of said main vectors. 

5. A method according to claim 4, wherein the spa- 
tial frequency spectrum is calculated based on a two- 
dimensional Fourier transform along at least two direc- 
tions in said image. 

30 6. A method according to claim 4 or 5, wherein the 

spatial frequency spectrum is calculated on the basis of 
a central part of the image. 

7. A method according to any one of claims 4-6, 
wherein the step of identifying at least two main vec- 
35 tors comprises the partial steps of 

localising in the spatial frequency spectrum posi- 
tions of peaks that exceed a given threshold value, and 
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selecting said at least two main vectors on the 
basis of said positions. 

8. A method according to any one of claims 4-7, 
wherein the steps of calculating a spatial frequency 

5 spectrum and identifying main vectors therein comprise 
the partial steps of 

changing the direction of a direction vector in 
steps within an angle range, 

calculating at least one absolute value of the two- 
10 dimensional Fourier transform for the image on the basis 
of each such direction vector, and 

identifying the absolute values that exceed said 
threshold value. 

9. A method according to claim 8, wherein the length 
15 of the direction vector is changed within a frequency 

range that comprises the nominal spatial frequency of the 
raster pattern (5) . 

10. A method according to claim 9, wherein the 
length of the direction vector is changed in steps, pre- 

20 ferably in steps that are inversely proportional to a 
power of 2 . 

11. A method according to any one of claims 8-10, 
wherein the position of each of said peaks is localised 
by calculation of the centre of gravity of the absolute 

25 values that exceed said threshold value and that are 

adjacent to each other in the spatial frequency spectrum. 

12. A method according to any one of claims 7-11, 
wherein the partial step of selecting at least two main 
vectors comprises 

30 letting each position identify a candidate vector 

(cl-c3) , 

letting at least one current image transform, which 
provides a given change in the relationship between two 
vectors, operate on said candidate vectors (cl-c3) , and 
35 selecting as main vectors the candidate vectors that 

provide a required mutual relationship for said at least 
one current image transform. 
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13. A method according to claim 12, wherein each 
current image transform corresponds to a given image 
relationship between a sensor (14) which records said 
image and an object (1) which is provided with said plu- 

5 rality of marks (4) . 

14. A method according to claim 12 or 13, comprising 
the steps of sequentially letting a series of different 
current image transforms operate on said candidate vec- 
tors (cl-c3), at least until a required mutual relation- 

10 ship is achieved between said candidate vectors. 

15. A method according to any one of claims 12-14, 
wherein said raster pattern (5) is identified on the 
basis of the image transform that gave rise to the 
required relationship between the candidate vectors 

15 (cl-c3) . 

16. A method according to claim 12 or 13, wherein 
the current image transform is selected on the basis of 
an earlier image transform that gave rise to the required 
relationship for a previous image. 

20 17. A method according to any one of claims 4-16, 

wherein said main vectors are selected on the basis of 
earlier main vectors that were determined for a previous 
image . 

18. A method according to any one of the preceding 
25 claims, comprising the step of transforming, said marks 

(4) with the main vectors as bases for producing a rota- 
tion-corrected image in which rotation of the marks (4) 
over the plane of the image is essentially eliminated. 

19. A method according to claim 18, comprising the 
30 additional step of compensating for perspective in the 

rotation-corrected image . 

20. A method according to claim 18 or 19, comprising 
the additional steps of 

determining the width of the peaks corresponding to 
35 the main vectors in a spatial frequency spectrum of said 
rotation-corrected image, and 
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compensating for perspective in the rotation-cor- 
rected image if the width exceeds a given width value. 

21. A method according to claim 19 or 20, wherein 
the step of compensating for perspective comprises the 
5 partial steps of 

measuring an inclination variation for the raster 
pattern along each main vector in the rotation-corrected 
image, 

calculating a perspective transform on the basis of 
10 the measured inclination variation, which perspective 

transform essentially eliminates said inclination varia- 
tion, and 

producing a perspective-corrected image by means of 
the perspective transform. 

15 22. A method according to claim 21, wherein the 

measurement of the inclination variation for the raster 
pattern along a selected main vector comprises the par- 
tial steps of 

via Fourier analysis of at least two subsets of the 

20 rotation-corrected image distributed along the selected 
main vector, calculating at least one subset main vector 
for each subset, 

identifying an initial position in the associated 
subset for each subset main vector, and 

25 calculating the inclination variation along the 

selected main vector on the basis of said subset main 
vectors and initial positions. 

23. A method according to claim 22, wherein the ini- 
tial position is identified on the basis of the centre of 

3 0 gravity of the marks incorporated in the respective sub- 
set . 

24. A method according to any one of claims 18-23, 
comprising the additional steps of 

measuring the phase displacement of the rotation- 
35 corrected or perspective-corrected image along the 
respective main vector via Fourier analysis of the 
rotation-corrected or perspective-corrected image, and 



WO 01/75783 PCT/SE01/00604 




38 



localising the raster pattern (5) relative to said 
marks (4) in the image on the basis of the measured phase 
displacements . 

25. A method according to claim 24, comprising the 
5 additional steps of 

calculating a normalising transform that places the 
intersections (6) of the raster pattern (5) a given dis- 
tance apart, and 

operating the normalising transform on the image in 
10 order to produce a normalised image. 

26. A method for identifying a virtual raster pat- 
tern (5) in an image of a plurality of marks (4), each of 
which is associated with a respective intersection (6) of 
raster lines (7) belonging to said raster pattern (5) , 

15 characterised by the steps of 

detecting main vectors of the image via Fourier ana- 
lysis, 

compensating for rotation in the plane of the image 
on the basis of said main vectors, 
20 detecting a perspective in the image, 

if necessary, compensating for said perspective, and 
identifying the virtual raster pattern (5) on the 
basis of said main vectors. 

27. A computer- readable computer program product 

25 which comprises a computer program with instructions for 
causing the computer to implement a method according to 
any one of claims 1-26. 

28. A device for position determination, comprising 
a sensor (14) for producing an image of a partial surface 

3 0 of a surface (2) which is provided with a position code 

in the form of a plurality of marks (4) , each of which is 
associated with one of a plurality of intersections (6) 
belonging to a virtual raster pattern (5) , and an image- 
processing means (16) which is arranged to calculate 

35 a position for the partial surface based on a subset of 
the surface (2) , the image-processing means (16) being 
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designed to identify the virtual raster pattern (5) in 
accordance with any one of claims 1-26. 

29. A device according to claim 28, which is hand- 
held. 

30. A device according to claim 2 8 or 29, which has 
a means (19) for wireless transmission of position infor- 
mation . 
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